Tell him the heart is willing but the flesh is weak. Suggest that some things have been done that 'stumble' you - but tell him in no uncertain terms that you do not want to discuss it at the moment. Get some time between you and meeting attendance. Let them forget about you over time. If they make efforts to restore you - just tell them you need more time to heal over the stumbling blocks.
I did not fade. I had to say what I knew. In my case it worked out pretty well since I have few family members in the cult. Some fade, others just leave. You have a chance to think it out, plan it if you like and then execute your plan.
Good luck.
Jeff